The present invention relates to software, and more specifically, to redirecting messages received from input devices.
In today""s multi-tasking environments, users generally interact with computers through graphical user interfaces. Typically, these graphical user interfaces display an application""s content within graphical windows on a display screen. Although multiple graphical windows can arise from a single program on a computer, the usual display screen includes windows from different programs executing independently of each other on the computer, or even several different computers.
With the development of faster computers having better graphical and multi-tasking capabilities, users may display a large number of windows on a desktop at any one time causing some windows on the display to overlap or obscure other windows. In order to view the obscured information, a user may resize or move the obstructing windows. This, however, may result in perhaps only seeing a portion of the desired information. Larger screen sizes and multiple monitors have been used to get around such a problem, but they are very expensive and can take up much desktop space. What is needed are new ways of interacting with windows in graphical user interfaces.
Typically, a user interacts with windows through various input devices, including a keyboard and a pointer device, such as a mouse. Many windowing systems provide mouse information to an application when the mouse pointer is located within an application""s window boundary. Additionally, many applications rely on the mouse pointer being within its boundaries when manipulating the window by moving, resizing, selecting menu items, and the like. For example, the user may open a menu by locating the mouse pointer over a graphical menu and selecting the menu. As long as the mouse pointer remains over the menu, the menu will be visible. If the mouse pointer is moved outside of the menu""s boundaries, however, the menu may disappear because the application assumes that the user is no longer interested in the menu.
Currently, there is no way of interacting with a graphical representation of a window as though the representation was a window object itself. While a user can apply transformations and effects to images on graphical displays, the user is not able to manipulate the graphical display through standard graphical user interface techniques. Accordingly, there is a need for a system and method for redirecting input so that a user may manipulate graphical images as if those images were window objects themselves.
The present invention is directed to providing a system and method for redirecting messages received from input devices. This redirection helps a user to interact with a graphical image as if the image itself was an actual application window. In one embodiment of the present invention, a window is redirected to appear as a texture map image on the user""s display, rather than as an actual window object. When a window is redirected, the actual window object is not displayed to the user. Instead, to the user, the texture map image appears and interacts as though it was the actual window object of the application that has been redirected while the actual window object is hidden from the user. The operating system, however, does not recognize a texture map image, i.e., a textured polygon as a window object, and therefore, input events that are directed toward the texture map image may not correspond with the actual location of the redirected window. Therefore, input messages received from input devices are redirected to correspond to the actual location of the window object of the redirected application. For example, when a window has its output redirected to a three-dimensional (3D) display, instead of being displayed in a standard graphical window environment, the input messages received within the 3D environment are transformed to correspond to the two-dimensional (2D) screen location of the actual window object. If the input messages are properly redirected to correspond to the actual window location, the user will be able to manipulate the texture map image as though it was the actual window. For example, the user can type in the 3D window, adjust the menus, and interact with the redirected output as though the application had not been redirected. If the input were not redirected, the 3D image would not behave as the application window. Instead, the input messages directed at the application may not even be processed by the application that has had a window redirected; or processed incorrectly. In summary, in accordance with this invention, messages arriving from input devices belonging to a particular application are redirected to correspond to the actual location where the application is located on the desktop.
One embodiment of the invention works in the following way. Applications are provided with the ability to set a style bit indicating that a window should be redirected. The application redirecting the window is referred to as the redirection host. Any time after the window has been created, the redirection host may set the style bit. Once the style bit has been set, the graphics device interface (GDI) visually removes the window from the desktop, creates an off-screen bitmap, and reroutes all further drawing operations applied to the window to the redirected location. Since the redirected application may not be aware of this change, the redirection host is responsible for propagating changes in the application""s visible appearance on the screen. To enable these changes, the GDI provides a global event (redirected paint) whenever the application has finished a visual update. By requesting a hook on this event, the redirection host obtains notification of the update, including which window was updated and the affected region within the window. Similarly, in order for the user to interact with the redirected output, input messages directed at the redirected window must often be transformed to correspond to the actual location of the window object. In one embodiment of the invention, a window test hook and get cursor position hook allows the operating system to intercept and change specified input messages to correspond to the actual screen location of the window object so that the application behaves as if it was not redirected.
In one embodiment of the invention, if a window has been redirected, the input messages obtained from the keyboard automatically go to the foreground application. The ability to bypass sending keyboard messages to the redirection host is achieved by setting a window flag to ensure that the redirection host does not become activated. Once the flag has been set the redirection host never receives keyboard messages because these are sent to the foreground application by the operating system. In one actual embodiment, therefore, the existing notions and policies of window activation and keyboard focus are consistent with the Windows operating system.
Mouse messages, on the other hand, typically get posted to the queue of the window that is located under the position of the cursor. Therefore, if the redirection style bit is detected as being set, a window hit test hook is installed that intercepts input messages before they are sent to the underlying window. If the mouse pointer is located over a redirected window""s texture map, the mouse coordinates are adjusted and the target window handle in the hit test structure is updated such that the pointer will appear to redirected window to be in the proper location. Once the input message has bee examined, and possibly transformed, the operating system posts the message to correct application. If the message was a mouse click over a texture map representing a redirected window, the application will become a foreground application and update its visuals and behavior accordingly. Thus, a click on a window object causes a corresponding application to receive activation and focus; a click on the 3-D scene background causes the redirection host to receive focus. This occurs even though neither mouse nor keyboard messages actually reach the host window event queue. If the input event is determined to be outside of a redirected window then the input event message is left alone and no changes are made.
Input messages relating to mouse position are also adjusted when a redirected window inspects the mouse position directly. Therefore, an additional hook is provided that intercepts direct inquiries to obtain the cursor position made by an application that has had a window redirected. This hook updates the mouse coordinates so as to correspond to the actual location of the redirected window if the cursor is over a position of the redirected window""s texture map.
In order for input redirection to properly work in one actual embodiment, each low-level mouse message generated in the system is inspected. Depending on how the output is redirected determines how difficult the hit test becomes. For example, when the output is redirected to a three-dimensional screen, the 3-D hit test can be non-trivial. It will appreciated, however, that deciding whether or not the pointer is over a pixel belonging to a redirected window can be accomplished in many different ways. For example, if the output is redirected to a 3-D scene, a 3-D hit test using hierarchical bounding boxes which returns a 2-D normalized coordinate on the face of the bounding box that was hit can be used. The normalized 2-D coordinate is then transformed into an appropriate pixel coordinate in the window""s coordinate system. A map of the screen locations of transformed 3-D objects representing windows can also be created, allowing a simple point in polygon test to determine whether the mouse is over that window. By inverting and caching the 3-D camera transform, the transformation to 2-D forged coordinates becomes much cheaper computationally than ray-plane intersection against a bounding box.